Modifying advertisement scores based on advertisement response probabilities

ABSTRACT

Advertisement response probabilities are utilized to alter advertisement scores. A plurality of possible advertisements is accessed from, for example, an advertisement database or advertisement pipeline. A response probability for each advertisement is determined. A response probability may be a probability that a user will “click,” or otherwise select an advertisement. Advertisements may be associated with probabilistic prediction models that take advertisement recipient attribute values as inputs and provide a probability distribution as output. A score associated with each of the possible advertisements is altered based on the response probability for each of the advertisements. Statistical prediction is used to determine how scores are to be altered. Advertisements with response probabilities less than a mean probability may have associated scores decreased. Conversely, advertisements with response probabilities greater than a mean probability may have associated scores increased.

BACKGROUND OF THE INVENTION

[0001] 1. The Field of the Invention

[0002] The present invention relates to targeted advertising. Morespecifically, the present invention relates to systems, methods, andcomputer program products for modifying an advertisement score based ona probability that a user will respond to the advertisement, theadvertisement score being indicative of whether the advertisement shouldbe presented.

[0003] 2. Background and Relevant Art

[0004] Advertisers often present advertisements to users of networkedcomputer systems (e.g., Internet-connected computer systems) in hopesthat the users of the networked computer systems will become interestedin the advertised products. At times, advertisers may presentadvertisements that are viewed by users and as a result generate userinterest in the advertised product. However, at other times, and perhapsmore frequently, viewed advertisements generate little, if any, userinterest in advertised products. In some cases, users simply ignoreadvertisements, not viewing them at all.

[0005] In the past, the reduced effectiveness of advertisementspresented on computer networks was in part due to advertisers havingreduced amounts of contextual data associated with possibleadvertisement recipients. In a broadcast or cable televisionenvironment, an advertiser may, at the very least, have contextual dataon the channel that will present an advertisement. In many cases, anadvertiser will also have contextual data on the programming and time ofday during which an advertisement will be presented. However, computernetworks, such as the Internet, may include voluminous amounts ofinformation, only a small portion of which may be of interest to aparticular user. An advertiser may have had no way to determine what aparticular user is interested in and thus present appropriateadvertisements.

[0006] As such, a variety of advertising techniques have been developedto “target” users on a computer network. These targeting techniques aredesigned to present advertisements that, if viewed, have increasedchances of generating user interest in an advertised product.Conventional targeting techniques often associate advertisements withadvertisement scores, where advertisements with higher scores arepresented to a user before advertisements with lower scores. Anadvertising server may generate a score for a number of advertisementsand then present the advertisements with the higher scores to a user.

[0007] An advertisement server may use deterministic rules whengenerating advertisement scores. Each advertisement may begin with abase score that is modified as successive rules are applied. Adeterministic rule may be, for example, “if a user is less than age 30,then increase the score for this advertisement.” The advertisementserver may access user data, for example, data contained in a userprofile, to determine how rules are applied. If the advertisement serveraccessed user data indicating that a particular user is age 25,application of the previous rule would result in an increase inassociated advertisement scores.

[0008] A series of rules may be applied based on different portions ofuser data, for example, age, sex, and income, to cause an advertisementscore for a particular group of users to be increased or decreased. Thisis beneficial, as an advertiser may configure a series of rules toincrease advertisement scores for particular groups of users theadvertiser believes are more likely to be interested in a particularproduct. Likewise, an advertiser may configure a series of rules todecrease advertisement scores for particular groups of users theadvertiser believes are less likely to be interested in a particularproduct.

[0009] Current targeting techniques are beneficial for increasing thechances of presenting advertisements that will generate user interest.However, current targeting techniques fail to consider the probabilitythat a potentially interested user will actually respond to anadvertisement by buying the advertised product or selecting theadvertisement (“clicking through”) to view additional information. Forexample, it may be that a user is interested in an advertised productbut for some reason has a decreased probability of responding to anadvertisement associated with the product. Presenting advertisements tousers who have decreased probabilities of responding to theadvertisements results in inefficient use of advertisement serverresources. Additionally, a user with a reduced probability forresponding to an advertisement may find presentation of such anadvertisement undesirable.

[0010] Therefore, what are desired are systems, methods, and computerprogram products, for modifying an advertisement score based on aprobability that a user will respond to the advertisement.

BRIEF SUMMARY OF THE INVENTION

[0011] The principles of the present invention provide for utilizingresponse probabilities, such as, for example, buying an advertisedproduct or selecting an advertisement to view additional information, tomodify a score that indicates whether or not to present theadvertisement to the user.

[0012] In accordance with the present invention, a number ofadvertisements are accessed. Advertisements may be accessed from a listor database of possible advertisements or may be received as input froman advertisement pipeline. Each advertisement is associated with aprobabilistic predictive model that maps a set of advertisementrecipient attribute-values to a response (or click) probability. In somecases, the probabilistic predictive model utilizes a decision tree,where each node in the decision tree is logically attached to one ormore other nodes. A root node is attached to other nodes (intermediatenodes and/or leaf nodes) that are directly beneath the root node.Intermediate nodes are attached to a node (root node or otherintermediate node) that is directly above the intermediate node and toother nodes (intermediate nodes and/or leaf nodes) that are directlybeneath the intermediate node. Leaf nodes are attached to a node (rootnode or intermediate node) that is directly above the leaf node.

[0013] Each root node and intermediate node may include decision logicthat causes another intermediate node or leaf node beneath the root nodeor intermediate node to be accessed. Decision logic may cause anothernode to be accessed based on user information, such as age, sex, oroccupation of a user. For example, a root node may include decisionlogic to access a first intermediate node if a user's age is less than18 and to access a second intermediate node if a user's age is 18 orgreater. Decision logic may be configured so that a series ofintermediate nodes are accessed before reaching a leaf node.

[0014] A response probability for each advertisement in the plurality ofpossible advertisements is determined. A response probability mayrepresent a probability that a user will buy a product or select anadvertisement by “clicking” on the advertisement. A decision tree may beutilized to determine response probabilities for advertisements.Starting at the root node and continuing through one or moreintermediate nodes, decision logic may analyze user informationassociated with a user, such as information from a user profile, andcause a leaf node to be accessed. Each leaf node may store a probabilityvalue between zero and one. Zero represents that a user will neverrespond to an advertisement and one represents that a user will alwaysrespond to an advertisement. A probability of .18, for example, mayrepresent an 18% chance that a user will respond to an advertisement.The accessed leaf node may include a value that represents theprobability that the user associated with the analyzed user informationwould respond to the advertisement.

[0015] For example, a user profile may contain the following informationfor a user: age-18, sex-male, and occupation-student. A decision treefor a particular advertisement may be utilized to determine theprobability that the user associated with the user profile would respondto the particular advertisement. Decision logic at a root node mayanalyze age information to cause one of a plurality of firstintermediate nodes to be accessed. For users who are age 18, aparticular first intermediate node may be accessed. Decision logic atthe particular first intermediate node may analyze sex information tocause one of a plurality of second intermediate nodes to be accessed.For users who are male, a particular second intermediate node may beaccessed. Decision logic at the particular second intermediate node mayanalyze occupation information to cause one of a plurality of leaf nodesto be accessed. For users who are students, a particular leaf node maybe accessed. The particular leaf node may include the probability thatan 18-year-old male student would respond to the particularadvertisement.

[0016] Based on the response probability for each of the possibleadvertisements, a score associated with each of the possibleadvertisements is altered. The score for each advertisement may be ascore that indicates whether or not to present the advertisement to theuser. A score may be received from an external module that is part of anadvertisement pipeline. When altering a score, the mean probability forresponding to an advertisement (i.e. a possible weighted average of theprobabilities of all the leaf nodes in a decision tree) may becalculated. The deviation of a particular leaf node's probability fromthe mean probability may be indicative of how a score is to be altered.

[0017] For example, if a particular probability of responding to anadvertisement is below the mean probability, a score may be multipliedby a value that decreases the score of the advertisement. Thus, thechance of presenting an advertisement to a user is decreased where theprobability that the user will respond to the advertisement is belowaverage. On the other hand, if a particular probability of responding toan advertisement is above the mean probability, a score may bemultiplied by a value that increases the score of the advertisement.Thus, the chance of presenting an advertisement to a user is increasedwhere the probability that the user will respond to the advertisement isabove average. Different ranges of deviation (e.g. some number ofstandard deviations from the mean) may result in differentmultiplicative factors being applied to a score. Multiplicative factorsfor altering a score may be calculated from values in a decision treeand/or may be user-configurable.

[0018] Modifying advertisement scores based on response probabilitiesincreases the chances of presenting advertisements a user will respondto and decreases the chances of presenting advertisements a user willnot respond to. This promotes conservation of resources in computersystems that present advertisements, as there is a decreased chance suchresources will be used to present advertisements that will not beresponded to. Further, the chances of presenting undesirableadvertisements to a user are also decreased.

[0019] Additional features and advantages of the invention will be setforth in the description that follows, and in part will be obvious fromthe description, or may be learned by the practice of the invention. Thefeatures and advantages of the invention may be realized and obtained bymeans of the instruments and combinations particularly pointed out inthe appended claims. These and other features of the present inventionwill become more fully apparent from the following description andappended claims, or may be learned by the practice of the invention asset forth hereinafter.

BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to describe the manner in which the above-recited andother advantages and features of the invention can be obtained, a moreparticular description of the invention briefly described above will berendered by reference to specific embodiments thereof which areillustrated in the appended drawings. Understanding that these drawingsdepict only typical embodiments of the invention and are not thereforeto be considered to be limiting of its scope, the invention will bedescribed and explained with additional specificity and detail throughthe use of the accompanying drawings in which:

[0021]FIG. 1 illustrates an example of a computer system that provides asuitable operating environment for the present invention.

[0022]FIG. 2 illustrates an example of some of the functional componentsthat may facilitate modifying advertisement scores based on responseprobabilities.

[0023]FIG. 3 is a flow diagram illustrating an example of a method formodifying advertisement scores based on response probabilities.

[0024]FIG. 4 illustrates an example of a decision tree.

[0025]FIG. 5 illustrates an example of response probability valuesdivided into different regions.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0026] The present invention extends to systems, methods, and computerprogram products for modifying advertisement scores based on responseprobabilities. A plurality of advertisements is accessed and a responseprobability for each advertisement in the plurality is determined. Anassociated advertisement score for each advertisement is modified basedon the corresponding response probability for each advertisement.

[0027] The embodiments of the present invention may comprise ageneral-purpose or special-purpose computer system including variouscomputer hardware components, which are discussed in greater detailbelow. Embodiments within the scope of the present invention alsoinclude computer-readable media for carrying or havingcomputer-executable instructions, computer-readable instructions, ordata structures stored thereon. Such computer-readable media may be anyavailable media, which is accessible by a general-purpose orspecial-purpose computer system. By way of example, and not limitation,such computer-readable media can comprise physical storage media such asRAM, ROM, EPROM, CD-ROM or other optical disk storage, magnetic diskstorage or other magnetic storage devices, or any other media which canbe used to carry or store desired program code means in the form ofcomputer-executable instructions, computer-readable instructions, ordata structures and which may be accessed by a general-purpose orspecial-purpose computer system.

[0028] In this description and in the following claims, a “network” isdefined as any architecture where two or more computer systems mayexchange data with each other.

[0029] When information is transferred or provided over a network oranother communications connection (either hardwired, wireless, or acombination of hardwired or wireless) to a computer system or computerdevice, the connection is properly viewed as a computer-readable medium.Thus, any such connection is properly termed a computer-readable medium.Combinations of the above should also be included within the scope ofcomputer-readable media. Computer-executable instructions comprise, forexample, instructions and data which cause a general-purpose computersystem or special-purpose computer system to perform a certain functionor group of functions.

[0030] In this description and in the following claims, a “computersystem” is defined as one or more software modules, one or more hardwaremodules, or combinations thereof, that work together to performoperations on electronic data. For example, the definition of computersystem includes the hardware components of a personal computer, as wellas software modules, such as the operating system of the personalcomputer. The physical layout of the modules is not important. Acomputer system may include one or more computers coupled via a computernetwork. Likewise, a computer system may include a single physicaldevice (such as a mobile phone or Personal Digital Assistant “PDA”)where internal modules (such as a memory and processor) work together toperform operations on electronic data.

[0031] Those skilled in the art will appreciate that the invention maybe practiced in network computing environments with many types ofcomputer system configurations, including personal computers, laptopcomputer, hand-held devices, multi-processor systems,microprocessor-based or programmable consumer electronics, network PCs,minicomputers, mainframe computers, mobile telephones, PDAs, pagers, andthe like. The invention may also be practiced in distributed computingenvironments where local and remote computer systems, which are linked(either by hardwired links, wireless links, or by a combination ofhardwired or wireless links) through a communication network, bothperform tasks. In a distributed computing environment, program modulesmay be located in both local and remote memory storage devices.

[0032]FIG. 1 and the following discussion are intended to provide abrief, general description of a suitable computing environment in whichthe invention may be implemented. Although not required, the inventionwill be described in the general context of computer-executableinstructions, such as program modules, being executed by computersystems. Generally, program modules include routines, programs, objects,components, data structures, and the like, which perform particulartasks or implement particular abstract data types. Computer-executableinstructions, associated data structures, and program modules representexamples of the program code means for executing steps of the methodsdisclosed herein. The particular sequences of such executableinstructions or associated data structures represent examples ofcorresponding acts for implementing the functions described in suchsteps.

[0033] With reference to FIG. 1, an example system for implementing theinvention includes a general-purpose computing device in the form ofcomputer system 120, including a processing unit 121, a system memory122, and a system bus 123 that couples various system componentsincluding the system memory 122 to the processing unit 121. Processingunit 121 may execute computer-executable instructions designed toimplement features of computer system 120, including features of thepresent invention. The system bus 123 may be any of several types of busstructures including a memory bus or memory controller, a peripheralbus, and a local bus using any of a variety of bus architectures. Thesystem memory includes read only memory (ROM) 124 and random accessmemory (RAM) 125. A basic input/output system (BIOS) 126, containing thebasic routines that help transfer information between elements withinthe computer 120, such as during start-up, may be stored in ROM 124.

[0034] The computer system 120 may also include a magnetic hard diskdrive 127 for reading from and writing to a magnetic hard disk 139, amagnetic disk drive 128 for reading from or writing to a removablemagnetic disk 129, and an optical disk drive 130 for reading from orwriting to removable optical disk 131 such as a CD-ROM or other opticalmedia. The magnetic hard disk drive 127, magnetic disk drive 128, andoptical disk drive 130 are connected to the system bus 123 by a harddisk drive interface 132, a magnetic disk drive-interface 133, and anoptical drive interface 134, respectively. The drives and theirassociated computer-readable media provide nonvolatile storage ofcomputer-executable instructions, data structures, program modules andother data for the computer system 120. Although the example environmentdescribed herein employs a magnetic hard disk 139, a removable magneticdisk 129 and a removable optical disk 131, other types of computerreadable media for storing data can be used, including magneticcassettes, flash memory cards, digital versatile disks, Bernoullicartridges, RAMs, ROMs, and the like.

[0035] Program code means comprising one or more program modules may bestored on the hard disk 139, magnetic disk 129, optical disk 131, ROM124 or RAM 125, including an operating system 135, one or moreapplication programs 136, other program modules 137, and program data138. A user may enter commands and information into the computer system120 through keyboard 140, pointing device 142, or other input devices(not shown), such as a microphone, joy stick, game pad, satellite dish,scanner, or the like. These and other input devices are often connectedto the processing unit 121 through a serial port interface 146 coupledto system bus 123. Alternatively, the input devices may be connected byother interfaces, such as a parallel port, a game port or a universalserial bus (USB). A monitor 147 or another display device is alsoconnected to system bus 123 via an interface, such as video adapter 148.In addition to the monitor, personal computers typically include otherperipheral output devices (not shown), such as speakers and printers.

[0036] The computer system 120 may operate in a networked environmentusing logical connections to one or more remote computers, such asremote computers 149A and 149B. Remote computers 149A and 149B may eachbe another personal computer, a server, a router, a network PC, a peerdevice, or other common network node, and typically include many or allof the elements described above relative to the computer system 120,although only memory storage devices 150A and 150B and their associatedapplication programs 136A and 136B are illustrated in FIG. 1. Thelogical connections depicted in FIG. 1 include a local area network(LAN) 151 and a wide area network (WAN) 152 that are presented here byway of example and not limitation. Such networking environments arecommonplace in office-wide or enterprise-wide computer networks,intranets and the Internet.

[0037] When used in a LAN networking environment, the computer system120 is connected to the local network 151 through a network interface oradapter 153. When used in a WAN networking environment, the computersystem 120 may include a modem 154, a wireless link, or other means forestablishing communications over the wide area network 152, such as theInternet. The modem 154, which may be internal or external, is connectedto the system bus 123 via the serial port interface 146. In a networkedenvironment, program modules depicted relative to the computer system120, or portions thereof, may be stored in the remote memory storagedevice. It will be appreciated that the network connections shown aremerely examples and other means of establishing communications over widearea network 152 may be used.

[0038] While FIG. 1 represents a suitable operating environment for thepresent invention, the principles of the present invention may beemployed in any system that is capable of, with suitable modification ifnecessary, implementing the principles of the present invention. Theenvironment illustrated in FIG. 1 is illustrative only and by no meansrepresents even a small portion of the wide variety of environments inwhich the principles of the present invention may be implemented.

[0039] In this description and in the following claims, a “logicalcommunication path” is defined as any communication path that may enablethe transport of electronic data between two entities such as computersystems or modules. The actual physical representation of acommunication path between two entities is not important and may changeover time. A logical communication path may include portions of a systembus, a local area network, a wide area network, the Internet,combinations thereof, or portions of any other path that may facilitatethe transport of electronic data. Logical communication paths mayinclude hardwired links, wireless links, or a combination of hardwiredlinks and wireless links. Logical communication paths may also includesoftware or hardware modules that condition or format portions of dataso as to make them accessible to components that implement theprinciples of the present invention. Such components may include, forexample, proxies, routers, firewalls, or gateways. Logical communicationpaths may also include portions of a Virtual Private Network (“VPN”).

[0040] In this description and in the following claims, a “response” isdefined as any action taken by an advertisement recipient that indicatesthe advertisement recipient has shown some interest in an advertisedproduct. Advertisement recipients may perform actions indicatingdifferent levels of interest in an advertised product. For example, anadvertisement recipient may show a higher level of interest in anadvertised product by buying the product. On the other hand anadvertisement recipient may show lower a level of interest by selectingan advertisement (e.g. “clicking through”) to view more informationabout an advertised product. It should be understood that response isdefmed, generally, to cover different levels of advertisement recipientinterest.

[0041] In accordance with the present invention, probabilitydetermination and score alteration modules as well as associated data,including user profiles and advertisements may be stored and accessedfrom any of the computer-readable media associated with computer system120. For example, portions of such modules and portions of associatedprogram data may be included in operating system 135, applicationprograms 136, other program modules 137 and/or program data 138, forstorage in system memory 124. Portions of such modules and associatedprogram data may also be stored in any of the mass storage devicespreviously described, for example hard disk 139. Execution of suchmodules may be performed in a distributed environment as previouslydescribed.

[0042]FIG. 2 illustrates some of the functional components that mayfacilitate modifying advertisement scores based on responseprobabilities. FIG. 2 includes advertising computer system 260 that maybe a flexible general-purpose computer system configured to implementthe principles of the present invention. As illustrated, advertisingcomputer system 260 includes probability determination module 261, whichmay be configured to determine response probabilities foradvertisements, and score alteration module 262, which may be configuredto alter advertisement scores based on response probabilities.

[0043]FIG. 3 is a flow diagram illustrating an example of a method formodifying advertisement scores based on a response probability. Themethod in FIG. 3 will be discussed with reference to the functionalcomponents included in FIG. 2.

[0044] The method in FIG. 3 may begin with a step for identifying aresponse probability for each advertisement in a plurality ofadvertisements. Response probabilities may be accessed from a databaseof response probabilities or may be received via an advertisementpipeline. Step 304 may include a corresponding act of accessing aplurality of possible advertisements (act 301). Possible advertisementsmay be accessed from an advertisement database or received via anadvertisement pipeline. Each advertisement in the possibleadvertisements may be associated with a probabilistic predictive model,such as, for example, decision trees, naive Bayes, or logisticalregression, that includes one or more response probabilities.

[0045] As shown in FIG. 2, possible advertisements 210 are received atprobability determination module 261 via logical communication path 271.Possible advertisements 210 represents a plurality of advertisementsthat may, if appropriate, be presented to an advertising recipient. Eachadvertisement in possible advertisements 210 includes a score and adecision tree, for example, advertisement 220 includes score 221 anddecision tree 222. Where three consecutive periods are illustrated inFIG. 2 (i.e. an ellipses), this represents that other advertisements mayprecede the illustrated advertisements or other advertisements followthe illustrated advertisements.

[0046] Logical communication path 271 may be a portion of anadvertisement pipeline. Possible advertisements 210 may have been outputfrom a previous module in the advertisement pipeline before beingreceived at probability determination module 210.

[0047] A score associated with an advertisement may be a numerical valuethat indicates whether or not an advertisement will be presented to auser. Advertisements associated with higher scores may be presentedbefore advertisements associated with lower scores. After scores areappropriately altered, a presentation module may present a specifiednumber of advertisements associated with higher scores, while otheradvertisement associated with lower scores are not presented. It shouldbe understood that the use of numerical values to determine whichadvertisements are presented is merely an example. It would be apparentto one skilled in the art, after having reviewed this description, thata variety of different scoring values may be utilized to determine whichadvertisements are presented.

[0048] A decision tree associated with an advertisement may include aroot node, one or more intermediate nodes, and one or more leaf nodes. Aroot node is attached to other nodes (intermediate nodes and/or leafnodes) that are directly beneath the root node. Intermediate nodes areattached to a node (root node or other intermediate node) that isdirectly above the intermediate node and to other nodes (intermediatenodes and/or leaf nodes) that are directly beneath the intermediatenode. Leaf nodes are attached to a node (root node or intermediate node)that is directly above the leaf node. Decision logic may be contained ateach root node and at each intermediate node and a response probabilityvalue may be contained at each leaf node. Response probability valuesrepresent a probability that an advertisement recipient will have aresponse to an advertisement. A response probability value may be anumeric value between zero and one. Zero may represent that anadvertisement recipient will never respond to (0%) an advertisement.Conversely, one may one may represent that an advertisement recipientwill always respond (100%) an advertisement. Response probability valuesmay be decimal numeric values representing some percentage chance thatan advertising recipient will respond to an advertisement. For example,a response probability value of 0.12 may represent a 12% chance that anadvertising recipient will respond to an advertisement.

[0049] Responding to an advertisement may result when an advertisingrecipient manipulates an input device such as, for example, a keyboardor mouse to provide an indication that they wish to view theadvertisement. “Clicking” on an advertisement with a mouse is one typeof response. An advertising recipient may also respond by selecting acurrently viewed advertisement when more information on an advertisedproduct is desired. Further, an advertising recipient may respond bypurchasing an advertised product. If a product is purchased “on-line”this information may be recorded into a database.

[0050] Shown in FIG. 4 is an example of a decision tree, decision tree400. Decision tree 400 includes root node 401, a plurality ofintermediate nodes, for example, intermediate nodes 412 and 445, eachrepresented by rectangles and a plurality of leaf nodes, for example,leaf nodes 433 and 454, each represented by circles. Root node 401, aswell as each intermediate node, contains decision logic that may causenodes below the root node or intermediate node to be accessed. Indecision tree 400, decision logic contained at root node 401 and atintermediate nodes is associated with observations about potentialadvertisement recipients.

[0051] It should be understood that the use of decision trees is merelyan example of a probabilistic prediction model. However, use of adecision tree is not important to practicing the present invention. Itwould be apparent to one skilled in the art, after having reviewed thisdescription, that a variety of different probabilistic prediction modelsmay be used to practice the present invention. The present invention maybe practiced with any model that takes advertisement recipientattributes and attribute values as inputs and provides a probabilitydistribution that an advertisement recipient will respond toadvertisements. Probabilistic prediction models include, for example,naive Bayes, logistic regression, generalized additive models, mixturemodels, and boosted versions of these classifiers. Likewise, it shouldbe understood that the illustrated decision tree configuration is one ofmany possible configurations. However, the illustrated decision treeconfiguration is not important to practicing the present invention. Itwould be apparent to one skilled in the art, after having reviewed thisdescription, that a variety of different decision tree configurations,including an inverted decision tree, may be used to practice the presentinvention.

[0052] Each advertisement in possible advertisements 210 may include adecision tree that is configured differently than the decision trees ofother advertisements. For example, the decision trees 222, 232, and 242may all be configured differently. Decision trees may be viewed ashaving different configurations when the nodes of the decision treescontain different decision logic and/or when the nodes of decision tressare arranged differently. Different decision tree configurations may bedesired for advertisements of different products. For example, a firstdecision tree configuration may be desirable for automotive products anda second decision tree configuration may desirable for householdproducts.

[0053] Step 304 may include a corresponding act of determining aresponse probability for each advertisement in the plurality of possibleadvertisements (act 302). For each advertisement, this may includetraversing a corresponding decision tree to access a responseprobability contained in a leaf node of the decision tree. Decisionlogic of different decision trees may be utilized for differentadvertisements. For advertisements 220, 230, and 240 this may includetraversing decision trees 222, 232, and 242 respectively. For example,for advertisement 230, decision tree 232 may be traversed to access aleaf node of decision tree 232.

[0054] Determining response probabilities may include referencing dataassociated with an advertising recipient (“recipient data”). Recipientdata may include demographic data associated with an advertisingrecipient such as, for example, age, income, sex, marital status, numberof children, etc. Recipient data may also include purchasing data suchas, for example, a list of products an advertising recipient recentlypurchased, when they purchased the products, what price was paid for theproducts, etc. Recipient data may also include business data such as,for example, an advertising recipient's type of business, place ofemployment, position, and membership in organizations, etc. Recipientdata may also include what web pages a recipient has accessed. It shouldbe understood that these are merely examples of the types of recipientdata that may be referenced. It would be apparent to one skilled in theart, after having reviewed this description, that a wide variety oftypes of recipient data, in addition to those described, may be utilizedto practice the present invention.

[0055] Recipient data may be referenced from a user profile thatcontains recipient data. As shown in FIG. 2, probability determinationmodule 261 may reference recipient data from user profile 250 vialogical communication path 272. Probability module 261 may utilize thedecision logic in decision trees along with the recipient data tocalculate a response probability for each advertisement.

[0056] Recipient data may be in the form of readable text that isincluded in a user profile. Readable text representing an example ofrecipient data will be described with reference to FIG. 4. In thereadable text example, when a sole period is encountered on threeconsecutive lines (i.e., a vertical ellipsis), this represents thatother recipient data may precede the illustrated readable text or mayfollow the illustrated readable text. Numbers enclosed in brackets areline numbers and are included for informational purposes to aid inclarifying the description of the readable text.

[0057] [1] Name-John Doe

[0058] [2] Age-35

[0059] [3] Status-Married

[0060] [4] Income- 43,000

[0061] [5] Children-1

[0062] [6] Employer-XYZ Corporation

[0063] [7] Position-Sales Manager

[0064] Decision tree 400 may be associated with one of theadvertisements include in possible advertisements 210. Probabilitydetermination module 261 may utilize the decision logic contained in theroot node and intermediate nodes of decision tree 400 along with thereadable text recipient data to access a response probability containedin a leaf node of decision tree 400.

[0065] Root node 401 of decision tree 400 contains decision logic thatmakes a decision based on an advertising recipient's yearly income, ifyearly income is less than $20,000, intermediate node 411 is accessed,if yearly income is between $20,000 and $50,000, intermediate node 412is accessed, and if yearly income is greater than $50,000, intermediatenode 413 is accessed. Line 4 of the recipient data includes the text“Income-43,000”, this may represent that an advertising recipient'syearly income is $43,000. The decision logic of root node 401 mayutilize this data to access intermediate node 412.

[0066] Intermediate node 412 contains decision logic that makes adecision based on an advertising recipient's age, if age is less than18, intermediate node 424 is accessed, if age is between 18 and 24,intermediate node 425 is accessed, if age is between 25 and 40,intermediate node 426 is accessed, and if age is greater than 40, leafnode 427 is accessed. Line 2 of the recipient data includes the text“Age-34”, this may represent that an advertising recipient is age 34.The decision logic of intermediate node 412 may utilize this data toaccess intermediate node 426.

[0067] Intermediate node 426 contains decision logic that makes adecision based on an advertising recipient's marital status, if single,leaf node 444 is accessed and if married, intermediate node 445 isaccessed. Line 3 of the recipient data ncludes the text“Status-Married”, this may represent that an advertising recipient ismarried. The decision logic of intermediate node 426 may utilize thisdata to access intermediate node 445.

[0068] Intermediate node 445 contains decision logic that makes adecision based on number of children, if three or less children leafnode 453 is accessed and if greater than three children leaf node 454 isaccessed. Line 5 of the recipient data includes the text “Children-1”,this may represent that an advertising recipient has one child. Thedecision logic of intermediate node 445 may utilize this data to accessleaf node 453.

[0069] Leaf node 453 contains a response probability of “0.06”. Thisresponse probability may represent a percentage that the advertisementassociated with decision tree 400 will be “clicked on” or otherwiseresponded to. This response probability may indicate that a 34 year old,married advertisement recipient with $44,000 yearly income and one childhas a 6% chance of responding to an advertisement associated withdecision tree 400.

[0070] Probability determination module 261 may utilize the readabletext recipient data (or may utilize other recipient data) and traverseother decision trees to access a response probability for eachadvertisement in possible advertisements 210. For example, decision tree222 may be traversed to access a response probability for advertisement220, decision tree 232 may be traversed to access a response probabilityfor advertisement 230, decision tree 242 may be traversed to access aresponse probability for advertisement 240, etc.

[0071] A score associated with each of the possible advertisements maybe altered based on the response probability for each of the possibleadvertisements (act 303). The scores may be the previously accessedscores, for example, scores 221, 231 and 232. As shown in FIG. 2, scorealteration module 262 may receive advertisements 210 along with responseprobabilities calculated by probability determination module 261 vialogical communication path 273. Score alteration module 262 maycalculate a statistical mean probability value by iterating over theresponse probabilities contained in every leaf node of every decisiontree associated with possible advertisements 210. The statistical meanvalue may represent an “average” response probability. The average canbe a weighted average, where the weight for a leaf node is equal to thepercentage of times that the leaf node is used to determine a responseprobability, or can be a simple average. If a non-decisiori-treepredictive model is used, such as, for example, naive Bayes orlogistical regression, advertising computer system 262 may trackresponse probabilities that are output and compute a simple average ofthose response probabilities. The deviation of a particular leaf node'sprobability from the mean probability may be indicative of how a scoreis to be altered. When simple averages are used the deviation may berepresented by a simple standard deviation. Likewise, when weightedaverages are used the deviation may be represented by a weightedstandard deviation.

[0072] For example, if a particular probability of responding to anadvertisement is below the mean probability, a score for anadvertisement may be decreased. Thus, the chance of presenting anadvertisement to a user may be decreased where the probability that theuser will respond to the advertisement is below average. On the otherhand, if a particular probability of responding to an advertisement isabove the mean probability, a score for an advertisement may beincreased. Thus, the chance of presenting an advertisement to a user maybe increased where the probability that the user will respond to theadvertisement is above average.

[0073] In some cases, advertisement scores may be increased or decreasedthrough the use of multiplicative factors. Different ranges of deviation(e.g. some number of standard deviations from the mean) may result indifferent multiplicative factors being applied to a score.Multiplicative factors for altering a score may be calculated fromvalues in a decision tree and/or may be user-configurable.

[0074] A group of pseudo-code instructions representing an example ofinstructions that may be utilized to perform score alteration will bedescribed. In the illustrated group of pseudo-code instructions, scorealteration is facilitated using simple averages and simple standarddeviations. However, it should be understood that weighted averages andweighted standard deviations may be used to facilitate score alteration.A module, for example, score alteration module 262, may executeinstructions similar to the group of pseudo-code instructions to alterthe scores of advertisements.

[0075] The following description is illustrative only. It would beapparent to one skilled in the art, after having reviewed thisdescription, that pseudo-code instructions may be implemented ascomputer-executable instructions using a wide variety of programminglanguages and programming techniques. In this pseudo-code description asole period on three consecutive lines (i.e., a vertical ellipsis)represents that other instructions may precede the illustratedinstructions or that other instructions may follow the illustratedinstructions. Numbers enclosed in brackets are line numbers and areincluded for informational purposes to aid in clarifying the descriptionof the instructions. Text preceded by two diagonal bars (“//”)represents informational comments describing the pseudo-code.[01]//means and stddevs are computed by iterating over each leaf node ofevery tree [02]A=mean (log(response probability)) −2*stddev(log(response probability)) [03]B=mean (log(responseprobability)) [04]C=mean (log(response probability)) +2*stddev(log(response probability)) [05] [06]min_factor=minimummultiplicative factor //set externally [07]max_factor=maximummultiplicative factor //set externally [08] [09]//p(ad=respond isshorthand for p(ad=respond|observations for user) [10]for each ad[11] if log(p(ad=respond)) >= C then [12] new_score(ad)=max_factor*old_score(ad) [13] elseif log(p(ad=respond)) <= A then[14] new_score(ad)=min_factor*old_score(ad) [15] elseiflog(p(ad=respond)) >B then [16] new_score(ad)=exp((log(p(ad=respond)) -B)/(C-B))*log(max_factor))* old_score(ad) [17] elseif log(p(ad=respond))< B then [18] new_score(ad)=exp((B -log(p(ad=respond)))/(B-A))*log(min_factor))*old_score(ad) [19] endif[20]endfor

[0076] At line 03 the variable “B” is set equal to a value representingthe statistical mean (hereinafter referred to as the “mean value”) ofthe response probability values in every leaf node of every decisiontree (hereinafter referred to as “all the response probability values”).At line 02 the variable “A” is set equal to a value representing themean value of all the response probability values minus two standarddeviations of all the response probability values. At line 04 thevariable “C” is set equal to a value representing the mean values of allthe response probability values plus two standard deviations of all theresponse probabilities. This essentially divides the range of all theresponse probability values into four regions. FIG. 5 is an example ofhow the values A, B, and C may divide a range of response probabilityvalues into different regions.

[0077] Shown in FIG. 5 is a range of response probabilities values withzero on the left and increasing to one on the right. In FIG. 5, zerorepresents a 0% chance that an advertisement will be responded to and 1represents a 100% chance that an advertisement will be responded to.Thus, the response probability for each advertisement may fall somewherewithin the range illustrated in FIG. 5.

[0078] Region 1 includes response probability values from zero up to andincluding A. Region 2 includes response probability values between A andB. Region 3 includes response probability values between B and C. Region4 includes response probability values from and including C up to 1. Thedistance between A and B is two standard deviations and the distancebetween B and C is two standard deviations. Thus, any responseprobability value at least two standard deviations less than B will fallin the region 1. Likewise, any response probability value at least twostandard deviations greater than C will fall in region 4. Any responseprobability value less than B, but not at least two standard deviationsless than B, will fall in the region 2. Any response probability valuegreater than B, but not at least two standard deviations greater than B,will fall in the region 3.

[0079] At line 06 the variable “min_factor” is set equal to a minimummultiplicative factor. This minimum multiplicative factor represents avalue that may be used to alter an advertisement score. The minimummultiplicative factor may be used to decrease advertisement scores thathave a response probability value in the region with the lowestprobability values (e.g. region 1 in FIG. 5). A user may configure theminimum multiplicative factor value externally. It may be thatmin_factor is set to a value less than one, thus reducing advertisementscores that are multiplied by min_factor.

[0080] At lines 07 the variable “max_factor” is set equal to a maximummultiplicative factor. This maximum multiplicative factor represents avalue that may be used to alter an advertisement score. The maximummultiplicative factor may be used to increase advertisement scores thathave a response probability value in the region with the highestprobability values (e.g. region 4 in FIG. 5). A user may configure themaximum multiplicative factor value externally. It may be thatmax_factor is set to a value greater than one, thus increasingadvertisement scores that are multiplied by max_factor.

[0081] The min_factor variable may be used to limit advertisement scorereduction. That is, no advertisement score may be decreased to a valuethat is less than a product of the score multiplied by min_factor. Themax_factor variable may be used to limit advertisement score increase.That is, no advertisement score may be increased to a value that is morethan a product of the score multiplied by max_factor. The min_factor andmax_factor variables may be included in intermediate values used toalter scores for advertisements that have response probability values inintermediate regions. For example, region 2 and region 3 in FIG. 5.

[0082] At line 10 the pseudo-code instruction “for each ad” indicatesthe beginning of a “for” loop that will be executed for eachadvertisement (e.g. each advertisement in possible advertisements 210).The pseudo-code instruction “endfor” at line 20 indicates the end of the“for” loop that begins at line 10. Taken together, lines 10 and 20indicate that the pseudo-code instructions from line 11 through line 19will be executed for each advertisement.

[0083] At line 11 through line 19 “p(ad=respond)” represents a responseprobability value for an advertisement. Such a response probabilityvalue may be a value that was contained in the leaf node of a decisiontree.

[0084] At line 11, it is determined if a response probability value foran advertisement is greater than or equal to C (i.e. at least twostandard deviations greater than B) and thus falls in region 4. When aresponse probability value falls in region 4, line 12 is executed. Atline 12, an advertisement score is altered. A variable “new_score” isset equal to the product of max_factor multiplied by a variable“old_score”. The variable old_score represents the score associated withan advertisement when the advertisement was initially received. Forexample, the scores 221, 231, and 241 as initially received byprobability determination module 261. The new_score variable representsa new score that will be associated with the advertisement and that willreplace the old_score. For example, altered score 223 may represent anew score for advertisement 220 and may replace score 221, whichrepresents an old score. Since max_factor may be set to a value greaterthan one, the value of new_score may be greater than the value ofold_score. Thus, an advertisement score may be increased when anassociated response probability value falls in region 4.

[0085] At line 13 it is determined if a response probability value foran advertisement is less than or equal to A (i.e. at least two standarddeviations less than B) and thus falls in region 3. When a responseprobability value falls in region 1, line 14 is executed. At line 14, anadvertisement score is altered. The variable new_score is set equal tothe product of min_factor multiplied by the variable old_score. Sincemin_factor may be set to a value less than one, the value of new_scoremay be less than the value of old_score. Thus, an advertisement scoremay be decreased when an associated response probability value falls inregion 1.

[0086] At line 15 it is determined if a response probability value foran advertisement is greater than B. Since all response probabilityvalues greater than or equal to C satisfy the “if” statement at line 11,line 15 essentially represents an “if” statement with the condition thata response probability value be greater than B and less than C. Suchresponse probability values would fall in region 3. When a responseprobability value falls in region 3, line 16 is executed. At line 16, anadvertisement score is altered. The variable new_score is set equal tothe product of the intermediate value[exp((log(p(ad=respond))-B)/(C-B))*log(max_factor))] multiplied by thevariable old_score. This essentially results in old_score beingmultiplied by an intermediate value between one and max_factor. Theintermediate value approaches one as a response probability valueapproaches B and the intermediate value approaches max_factor as aresponse probability value approaches C.

[0087] At line 16, the value of ((log(p(ad=respond)-B)/(C-B)) decreasesas a response probability value approaches B. If a response probabilityvalue were to equal B, the value of ((log(p(ad=respond)-B)/(C-B)) wouldequal zero (i.e. ((B-B)/(C-B))). This results in an intermediate valueof exp(0*log(max_factor)), which equals one. Thus, if a responseprobability for an advertisement were to equal B, new_score would equalthe product of one multiplied by old_score.

[0088] On the other hand, the value of ((log(p(ad=respond)-B)/(C-B))increases as a response probability value approaches C. If a responseprobability value were to equal C, the value of((log(p(ad=respond)-B)/(C-B)) will equal one (i.e. ((C-B)/(C-B))). Thisresults in an intermediate value of exp(1*log(max_factor)), which equalsmax_factor. Thus, if a response probability for an advertisement were toequal C, new_score would equal the product of max_factor multiplied byold score.

[0089] Thus, an advertisement score may be increased when an associatedresponse probability value falls in region 3. However, it may be thatthe magnitude of an increase is less than the magnitude of an increasewhen response probability values fall in region 4.

[0090] At line 17 it is determined if a response probability value foran advertisement is less than B. Since all response probability valuesless than or equal to A satisfy the “if” statement at line 13, line 18essentially represents an “if” statement with the condition that aresponse probability value be less than B and greater than A. Suchresponse probability values would fall in region 2. When a responseprobability value falls in region 2, line 18 is executed. At line 18, anadvertisement score is altered. The variable new_score is set equal tothe product of [exp((B-log(p(ad=respond)))/(B-A))*log(min_factor))]multiplied by the variable old_score. This essentially results inold_score being multiplied by an intermediate value between min_factorand 1. The intermediate value approaches min_factor as a responseprobability value approaches A and the intermediate value approaches 1as a response probability value approaches B.

[0091] The value of ((B-log(p(ad=respond)))/(B-A)) decreases as aresponse probability value approaches B. If a response probability valuewere to equal B, the value of ((B-log(p(ad=respond)))/(B-A)) would equalzero (i.e. ((B-B)/(B-A))). This results in an intermediate value ofexp(0*log(min_factor)), which equals one. Thus, if a responseprobability for an advertisement were to equal B, new score would equalthe product of one multiplied by old_score.

[0092] On the other hand, the value of ((B-log(p(ad=respond)))/(B-A))increases as a response probability value approaches A. If a responseprobability value were to equal A, the value of((B-log(p(ad=respond)))/(B-A)) would equal 1 (i.e. ((B-A)/(B-A))). Thisresults in an intermediate value of exp(1*log(min factor)), which equalsmin_factor. Thus, if a response probability for an advertisement were toequal A, new score would equal the product of min_factor multiplied byold_score.

[0093] Thus, an advertisement score to may be decreased when anassociated response probability value falls in region 2. However, it maybe that the magnitude of a decrease is less than the magnitude of adecrease when response probability values fall in region 1.

[0094] As illustrated by the pseudo-code instructions, there is no setof conditions for altering an advertisement response probability with avalue equal to B. This may indicate a desire not to modify scores foradvertisements that have an “average” (either simple or weighted) chanceof being responded to.

[0095] In should be understood that the described pseudo-codeinstructions and the illustrated response probability regions in FIG. 5are merely examples. It would be apparent to one skilled in the art,after having reviewed this description that a variety of differentpseudo-code instructions may be implemented to create a variety ofdifferent response probability regions.

[0096] Advertisements that include altered scores may be output. Shownin FIG. 2, score alteration module 262 outputs possible advertisements210 via logical communication path 274. Advertisements may be output toan advertisement database or to a module that is included in anadvertisement pipeline. Logical communication path 274 may represent aportion of an advertisement pipeline.

[0097] In one alternate embodiment, the present invention is practicedto modify scores for content, such as, for example, content on the WorldWide Web (“WWW”), based on response probabilities. A plurality ofpossible portions of Web content (e.g. Web pages) are accessed. Eachportion of Web content is associated with a probabilistic predictionmodel. A response probability is determined for each portion of Webcontent in the plurality of possible portions of Web content. Respondingto Web content includes any action taken by a recipient of the Webcontent that indicates the Web content recipient has shown some interestin the subject matter of the Web content. Similar to advertisements,this may include selecting a portion of Web content (e.g. a Web page) byclicking through to view more information about the subject matter ofthe Web content. Based on the response probability for each of thepossible portions of Web content, a score associated with each of thepossible portions of Web content are altered.

[0098] Modifying advertisement scores based on response probabilitiesincreases the chances of presenting advertisements a user will respondto and decreases the chances of presenting advertisements a user willnot respond to. This promotes conservation of resources in computersystems that present advertisements, as there is a decreased chance suchresources will be used to present advertisements that will not beresponded to. Further, the chances of presenting undesirableadvertisements to a user are also decreased.

[0099] The present invention may be embodied in other specific formswithout departing from its spirit or essential characteristics. Thedescribed embodiments are to be considered in all respects only asillustrative and not restrictive. The scope of the invention is,therefore, indicated by the appended claims rather than by the foregoingdescription. All changes, which come within the meaning and range ofequivalency of the claims, are to be embraced within their scope.

What is claimed and desired secured by United States Letters Patent is:
 1. In a network environment that includes at least an advertisement computer system, a method for targeting online advertisements to users based on the probability that a user will respond to possible advertisements, comprising the following: an act of accessing a plurality of possible advertisements; an act of determining a response probability for each advertisement in the plurality of possible advertisements, the response probability representing a probability that a user will respond to an advertisement; and an act of altering a score associated with each of the possible advertisements based on the response probability for each of the possible advertisements, the score being used to indicate whether or not to present the advertisement to the user.
 2. The method as recited in claim 1, wherein the act of accessing a plurality of possible advertisements comprises the following: an act of accessing a plurality of possible advertisements from an advertisement database.
 3. The method as recited in claim 1, wherein the act of accessing a plurality of possible advertisements comprises the following: an act of accessing a plurality of possible advertisements from an advertisement pipeline.
 4. The method as recited in claim 1, wherein the act of determining a response probability for each advertisement in the plurality of possible advertisements c omprises the following: an act of using a probabilistic prediction model to determine a response probability for each advertisement in the plurality of possible advertisements.
 5. The method as recited in claim 4, wherein the act of using a probabilistic prediction model to determine a response probability for each advertisement in the plurality of possible advertisements comprises the following: an act of using a decision tree to determine a response probability for an advertisement.
 6. The method as recited in claim 5, wherein the act of using a decision tree to determine a response probability for an advertisement comprises the following: an act of traversing the decision tree to reach a leaf node that contains a response probability.
 7. The method as recited in claim 4, wherein the act of using a probabilistic prediction model to determine a response probability for each advertisement in the plurality of possible advertisements comprises the following: an act of using a naive Bayes predictive model to determine a response probability for each advertisement in the plurality of possible advertisements.
 8. The method as recited in claim 4, wherein the act of using a probabilistic prediction model to determine a response probability for each advertisement in the plurality of possible advertisements comprises the following: an act of using a logistical regression predictive model to determine a response probability for each advertisement in the plurality of possible advertisements.
 9. The method as recited in claim 1, wherein the act of determining a response probability for each advertisement in the plurality of possible advertisements comprises the following: an act of using recipient attribute values to determine a response probability for each advertisement in the plurality of possible advertisements.
 10. The method as recited in claim 1, wherein the act of determining a response probability for each advertisement in the plurality of possible advertisements comprises the following: an act of using a probability distribution to determine a response probability for each advertisement in the plurality of possible advertisements.
 11. The method as recited in claim 1, wherein the act of determining a response probability for each advertisement in the plurality of possible advertisements comprises the following: an act of determining a probability that a user will click on an advertisement.
 12. The method as recited in claim 1, wherein the act of determining a response probability for each advertisement in the plurality of possible advertisements comprises the following: an act of determining a probability that a user will buy an advertised product.
 13. The method as recited in claim 1, wherein the act of altering a score associated with each of the possible advertisements based on the response probability for each of the possible advertisements comprises the following: an act of increasing scores associated with advertisements that have a response probability greater than a designated value.
 14. The method as recited in claim 13, wherein the act of increasing scores associated with advertisements that have a response probability greater than a designated value comprises the following: an act of increasing scores associated with advertisements that have a response probability greater than the mean value of the response probabilities included in a probability distribution.
 15. The method as recited in claim 14, wherein the act of increasing scores associated with advertisements that have a response probability greater than the mean value of the response probabilities included in a probability distribution comprises the following: an act of increasing scores that have a response probability greater than a weighted average of the response probabilities included in a probability distribution.
 16. The method as recited in claim 14, wherein the act of increasing scores associated with advertisements that have a response probability greater than the mean value of the response probabilities included in a probability distribution comprises the following: an act of increasing scores associated with advertisements that have a response probability greater than the mean value of the response probabilities included in all the leaf nodes of decision trees associated with each of the possible advertisements.
 17. The method as recited in claim 14, wherein the act of increasing scores associated with advertisements that have a response probability greater than the mean value of the response probabilities included in a probability distribution comprises the following: an act of increasing a first set of scores associated with advertisements that have a response probability greater than the mean value plus a specified number of standard deviations of the response probabilities in a probability distribution, by multiplying the scores by a first multiplicative factor.
 18. The method as recited in claim 17, wherein the act of increasing scores associated with advertisements that have a response probability greater than the mean value of the response probabilities included in a probability distribution comprises the following: an act of increasing a second set of scores associated with advertisements that have a response probability less than the mean value plus a specified number of standard deviations of the response probabilities in the probability distribution, by multiplying the scores by a second multiplicative factor.
 19. The method as recited in claim 18, wherein the first multiplicative factor has a magnitude greater than the second multiplicative factor.
 20. The method as recited in claim 14, wherein the act of increasing scores associated with advertisements that have a response probability greater than the mean value of the response probabilities included in a probability distribution comprises the following: an act of increasing scores associated with advertisements that have a response probability greater than the mean value of the response probabilities included in a probability distribution plus a specified number of weighted standard deviations of the response probabilities included in the probability distribution.
 21. The method as recited in claim 14, wherein the act of increasing scores associated with advertisements that have a response probability greater than the mean value of the response probabilities included in a probability distribution comprises the following: an act of increasing scores that have a response probability greater than a mean value of the response probabilities included in a probability distribution that was output from a naive Bayes predictive model.
 22. The method as recited in claim 1, wherein the act of altering a score associated with each of the possible advertisements based on the response probability for each of the possible advertisements comprises the following: an act of decreasing scores associated with advertisements that have a response probability less than a designated value.
 23. The method as recited in claim 22, wherein the act of decreasing scores associated with advertisements that have a response probability less than a designated value comprises the following: an act of decreasing scores associated with advertisements that have a response probability less than the mean value of the response probabilities included in a probability distribution.
 24. The method as recited in claim 23, wherein the act of decreasing scores associated with advertisements that have a response probability less than the mean value of the response probabilities included in a probability distribution comprises the following: an act of decreasing scores that have a response probability greater than a weighted average of the response probabilities included in a probability distribution.
 25. The method as recited in claim 23, wherein the act of decreasing scores associated with advertisements that have a response probability less than the mean value of the response probabilities included in a probability distribution comprises the following: an act of decreasing scores associated with advertisements that have a response probability less than the mean value of the response probabilities included in all the leaf nodes of decision trees associated with each of the possible advertisements.
 26. The method as recited in claim 23, wherein the act of decreasing scores associated with advertisements that have a response probability less than the mean value of the response probabilities included in a probability distribution comprises the following: an act of decreasing a first set of scores associated with advertisements that have a response probability less than the mean value minus a specified number of standard deviations of the response probabilities in a probability distribution, by multiplying the scores by a third multiplicative factor.
 27. The method as recited in claim 26, wherein the act of decreasing scores associated with advertisements that have a response probability less than the mean value of the response probabilities included in a probability distribution comprises the following: an act of decreasing a second set of scores associated with advertisements that have a response probability greater than the mean value minus a specified number of standard deviations of the response probabilities in a probability distribution, by multiplying the scores by a fourth multiplicative factor.
 28. The method as recited in claim 27, wherein the third multiplicative factor has a magnitude less than the fourth multiplicative factor.
 29. The method as recited in claim 23, wherein the act of decreasing scores associated with advertisements that have a response probability less than the mean value of the response probabilities included in a probability distribution comprises the following: an act of decreasing scores associated with advertisements that have a response probability less than the mean value of the response probabilities included in a probability distribution plus a specified number of weighted standard deviations of the response probabilities included in the probability distribution.
 30. The method as recited in claim 23, wherein the act of decreasing scores associated with advertisements that have a response probability less than the mean value of the response probabilities included in a probability distribution comprises the following: an act of decreasing scores that have a response probability less than a mean value of the response probabilities included in a probability distribution that was output from a logistical regression predictive model.
 31. The method as recited in claim 1, wherein the act of altering a score associated with each of the possible advertisements based on the response probability for each of the possible advertisements comprises the following: an act of using statistical prediction to alter a score associated with each of the possible advertisements.
 32. The method as recited in claim 1, wherein the act of altering a score associated with each of the possible advertisements based on the response probability for each of the possible advertisements comprises the following: an act of altering a score associated with an advertisement based on the probability that a user will click on the advertisement.
 33. The method as recited in claim 1, further comprising: an act of associating an altered scored with each of the possible advertisements.
 34. The method as recited in claim 33, further comprising: an act of outputting each of the possible advertisements.
 35. The method as recited in claim 34, wherein an act of outputting each of the possible advertisements comprises the following: an act of outputting each of the possible advertisements to an advertisement database.
 36. The method as recited in claim 34, wherein an act of outputting each of the possible advertisements comprises the following: an act of outputting each of the possible advertisements to an advertisement pipeline.
 37. In a network environment that includes at least an advertisement computer system, a method for targeting online advertisements to users based on the probability that a user will select possible advertisements, comprising the following: a step for identifying a response probability for each advertisement in a plurality of possible advertisements; and an act of altering a score associated with each of the possible advertisements based on the response probability for each of the possible advertisements, the score being used to indicate whether or not to present the advertisement to the user.
 38. A computer program product for use in a network environment that includes at least an advertisement computer system, the computer program product for implementing a method for targeting online advertisements to users based on the probability that a user will respond to possible advertisements, the computer program product comprising one or more computer-readable media having stored thereon the following: computer-executable instructions for accessing a plurality of possible advertisements; computer-executable instructions for determining a response probability for each advertisement in the plurality of possible advertisements, the response probability representing a probability that a user will respond to an advertisement; and computer-executable instructions for altering a score associated with each of the possible advertisements based on the response probability for each of the possible advertisements, the score being used to indicate whether or not to present the advertisement to the user.
 39. The computer program product as recited claim 38, wherein the one or more computer-readable media include physical storage media.
 40. The computer program product as recited claim 38, wherein the one or more computer-readable media include system memory.
 41. A network system, comprising: a user computer system configured to present advertisements to a user; and an advertisement computer system that is network connectable to the user computer system and that is configured to access a plurality of advertisements, to determine a response probability for each advertisement in the plurality of advertisements, and to alter a score for each advertisement based on the response probability for each advertisement.
 42. In a network environment that includes at least a Web content computer system, a method for targeting Web content to users based on the probability that a user will respond to possible Web content, comprising the following: an act of accessing a plurality of possible portions of Web content; an act of determining a response probability for each portion of Web content in the plurality of possible portions of Web content, the response probability representing a probability that a user will respond to a portion of Web content; and an act of altering a score associated with each of the possible portions of Web content based on the response probability for each of the possible portions of Web content, the score being used to indicate whether or not to present the portion of Web content to the user.
 43. The method as recited in claim 42, wherein the act of accessing a plurality of possible portions of Web content comprises the following: an act of accessing a plurality of Web pages.
 44. The method as recited in claim 42, wherein the act of determining a response probability for each portion of Web content in the plurality of possible portions of Web content comprises the following: an act of using a probabilistic prediction model to determine a response probability for each portion of Web content in the plurality of possible portions of Web content. 